Coding apparatus, coding method, decoding apparatus, and decoding method

ABSTRACT

A coding apparatus for performing coding of an image including a plurality of tiles includes determining means for determining, with respect to a plurality of boundaries constituted by the plurality of tiles, whether or not filter processing is performed on pixels adjacent to the boundaries, and coding means for performing coding of control information indicating whether or not the filter processing is performed on the pixels adjacent to the boundaries in accordance with a determining by the determining means with respect to at least two of the plurality of boundaries.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Patent ApplicationNo. PCT/JP2017/044662, filed Dec. 13, 2017, which claims the benefit ofJapanese Patent Application No. 2016-249173, filed Dec. 22, 2016, bothof which are hereby incorporated by reference herein in their entirety.

TECHNICAL FIELD

The present invention relates to a coding apparatus, a coding method anda program, a decoding apparatus, a decoding method and a program, andcoded image data, and particularly relates to filtering processing on ablock boundary.

BACKGROUND ART

As a coding method for compression recording of a moving image, an HEVC(High Efficiency Video Coding) coding method (hereinafter, referred toas HEVC) has been known. A technique called Tile (tiles) with which oneframe is divided into a plurality of rectangular areas to enableparallel processing for coding and decoding and the like is adopted forthe HEVC. When the tiles are used, an increase in speed based on theparallel processing for the coding and the decoding can be realized, andalso memory capacities included in an image coding apparatus and animage decoding apparatus can be reduced.

In addition, to improve an image quality of the image after the codingin the HEVC, in-loop filter processing such as deblocking filter orsample adaptive offset is adopted. Although such in-loop filterprocessing can also be applied to pixels striding a tile boundary, whenthe in-loop filter processing is applied to the pixels striding the tileboundary, an issue may be caused on the parallel processing for thecoding and the decoding in some cases. For this reason, a syntax elementof loop_filter_across_tiles_enabled_flag with which it is possible tochoose whether or not the in-loop filter processing is applied to thepixels striding the tile boundary is adopted in the HEVC. That is, it isdesigned that, in a case where the above-described syntax element is 1,the application of the in-loop filter to the tile boundary is enabled,and in the case of 0, the application of the in-loop filter to the tileboundary is prohibited. According to this, the above-described syntaxelement can be set as 0 in a case where an importance is placed onparallel implementation, and the above-described syntax element can beset as 1 in a case where an importance is placed on an image quality onthe tile boundary.

In recent years, a use case where a 360° video is captured by aplurality of cameras, and the captured image is subjected to compressionand coding has been generated along with a development in a VR (VirtualReality) technology. As a capturing method for the 360° video, atechnique for capturing images in respective directions of top, bottom,left, right, front, and back by six cameras to be combined with oneanother has been proposed (NPL 1). The thus captured images are set asone image by rearranging the captured six images to be combined with oneanother and subjected to the compression and the coding. With regard tothe rearrangement, a technique for arranging so as to develop a die asillustrated in FIG. 9A and a technique for further rearranging into arectangle such that the area of the image after the combination becomesthe smallest as illustrated in FIGS. 9B to 9D have been under review(NPL 2). According to the technique as illustrated in FIG. 9A,boundaries of adjacent images such as left and front, and front andright are regularly continuous, but wasteful areas are generated in fourcorners of the image after the combination. On the other hand, accordingto the technique as illustrated in FIGS. 9B to 9D, the wasteful areasare not generated in the image after the combination, but a continuousboundary and a discontinuous boundary are mixed among the boundaries ofthe adjacent images.

CITATION LIST Non Patent Literature

-   NPL 1: JVET Contribution JVET-00021 the Internet    <http://phenix.int-evry.fr/jvet/doc_end_user/documents/3_Geneva/wg11/>-   NPL 2: JVET Contribution JVET-D0022 the Internet    <http://phenix.int-evry.fr/jvet/doc_end_user/documents/4_Chengdu/wg11/>

SUMMARY OF INVENTION

It is conceivable that use of the in-loop filter represented by theabove-described deblocking filter or sample adaptive offset is effectivefrom the viewpoint of coding efficiency. In addition, in a case wherethe image obtained by the combination after the capturing by theplurality of cameras as illustrated in FIGS. 9B to 9D described above,that is, the image in which the continuous boundary and thediscontinuous boundary are mixed among the boundaries in the image issubjected to coding, the coding is naturally performed while the imagescaptured by the respective cameras are associated with tiles. However,according to the existing technique represented by the HEVC, it is onlypossible to choose whether or not the in-loop filter processing isuniformly applied to all the tile boundaries in the image. That is, itis only possible to choose whether or not the in-loop filter is appliedwithout distinguishing the continuous tile boundary and thediscontinuous tile boundary from each other. In the above-describedcase, when non-application of the in-loop filter is chosen byprioritizing the discontinuous tile boundary, the in-loop filter cannotbe applied to the continuous tile boundary where the image qualityimprovement can be normally expected by the in-loop filter processing.On the other hand, when application of the in-loop filter is chosen byprioritizing the continuous tile boundary, the in-loop filter is appliedto the discontinuous tile boundary, and an unnecessary image qualitydeterioration is caused in a surrounding of the discontinuous tileboundary. Therefore, the present invention has been made to address theabove-described issue and aims at enabling adaptive application of thein-loop filter processing to the tile boundary while a continuity of thetile boundary is taken into account.

To address the above-described issue, a coding apparatus according tothe present invention has the following configuration. That is, thecoding apparatus for performing coding of an image including a pluralityof tiles includes determining means for determining, with respect to aplurality of boundaries constituted by the plurality of tiles, whetheror not filter processing is performed on pixels adjacent to theboundaries, and coding means for performing coding of controlinformation indicating whether or not the filter processing is performedon the pixels adjacent to the boundaries in accordance with adetermining by the determining means with respect to at least two of theplurality of boundaries.

Furthermore, a decoding apparatus according to the present invention hasthe following configuration. That is, the decoding apparatus fordecoding an image including a plurality of tiles from a bit streamincludes decoding means for decoding the image, generation means forgenerating, with respect to a plurality of boundaries constituted by theplurality of tiles, control information indicating whether or not filterprocessing is performed on pixels adjacent to the boundaries from thebit stream, determining means for determining whether or not thefiltering processing is performed on the pixels adjacent to at least aplurality of boundaries in accordance with the control informationgenerated by the generation means, and processing means for performingthe filtering processing with respect to the boundaries where thefiltering processing is performed which is determined by the determiningmeans.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagecoding apparatus.

FIG. 2 is a block diagram illustrating a configuration of an imagedecoding apparatus.

FIG. 3A illustrates an example of a relationship between a continuity ofa tile boundary and target pixels of in-loop filter processing.

FIG. 3B illustrates an example of the relationship between thecontinuity of the tile boundary and the target pixels of the in-loopfilter processing.

FIG. 4 is a flow chart illustrating image coding processing in the imagecoding apparatus.

FIG. 5 is a flow chart illustrating the in-loop filter processing in theimage coding apparatus and the image decoding apparatus.

FIG. 6A illustrates an example of a bit stream structure generated bythe image coding apparatus and decoded by the image decoding apparatus.

FIG. 6B illustrates an example of the bit stream structure generated bythe image coding apparatus and decoded by the image decoding apparatus.

FIG. 7 is a flow chart illustrating decoding processing in the imagedecoding apparatus.

FIG. 8 is a block diagram illustrating a hardware configuration exampleof a computer that can be applied to the image coding apparatus and theimage decoding apparatus.

FIG. 9A illustrates an arrangement example in which images captured by aplurality of cameras are rearranged to obtain a single image.

FIG. 9B illustrates an arrangement example in which the images capturedby the plurality of cameras are rearranged to obtain the single image.

FIG. 9C illustrates an arrangement example in which the images capturedby the plurality of cameras are rearranged to obtain the single image.

FIG. 9D illustrates an arrangement example in which the images capturedby the plurality of cameras are rearranged to obtain the single image.

FIG. 10A represents a syntax related to tile division information andfilter control information of a bit stream generated by the image codingapparatus and decoded by the image decoding apparatus.

FIG. 10B represents the syntax related to the tile division informationand the filter control information of the bit stream generated by theimage coding apparatus and decoded by the image decoding apparatus.

FIG. 10C represents the syntax related to the tile division informationand the filter control information of the bit stream generated by theimage coding apparatus and decoded by the image decoding apparatus.

DESCRIPTION OF EMBODIMENTS

Hereinafter, the present invention will be described in detail by way ofits preferred embodiments with reference to the accompanied drawings. Itshould be noted that configurations illustrated in the followingembodiments are merely examples, and the present invention is notlimited to the illustrated configurations.

Hereinafter, embodiments of the present invention on a coding side willbe described by using the drawings. According to the present embodiment,in particular, a case where an image illustrated in FIG. 9B is input toperform coding will be described.

FIG. 1 is a block diagram illustrating an image coding apparatusaccording to the present embodiment. All of respective blocks in FIG. 1may be constituted as hardware, or part or all of the blocks may also beconstituted as software.

In FIG. 1, a terminal 101 is a terminal to which image data is input.

A tile division unit 102 determines a tile division method for the inputimage and performs division processing.

A filter control information generation unit 103 generates and outputsfilter control information corresponding to information on whether ornot in-loop filter processing which will be described below is performedwith respect to the pixels on the respective tile boundaries.

A prediction unit 104 performs intra prediction corresponding tointra-frame prediction, inter prediction corresponding to inter-frameprediction, or the like with respect to image data in units of blocks togenerate prediction image data. Furthermore, a prediction error iscalculated from the input image data and the prediction image data to beoutput. In addition, information required for the prediction such as,for example, information of a prediction mode is also output togetherwith the prediction error. Hereinafter, this information required forthe prediction will be referred to as prediction information.

A transformation and quantization unit 105 performs orthogonaltransformation of the prediction error in units of blocks to obtain atransformation coefficient and further performs quantization to obtain aquantization coefficient.

An inverse quantization and inverse transformation unit 106 performsinverse quantization of the quantization coefficient output from thetransformation and quantization unit 105 to reproduce the transformationcoefficient and further performs inverse orthogonal transformation toreproduce the prediction error.

A frame memory 108 is a memory that stores the reproduced image data.

An image reproduction unit 107 refers to the frame memory 108 asappropriate in accordance with the prediction information output fromthe prediction unit 104 to generate the prediction image data andgenerates reproduction image data from the generated prediction imagedata and the input prediction error to be output.

An in-loop filter unit 109 performs in-loop filter processing such asthe deblocking filter or the sample adaptive offset with respect to thereproduction image and outputs the image on which the filter processinghas been performed.

A coding unit 110 performs coding of the quantization coefficient outputfrom the transformation and quantization unit 105 and the predictioninformation output from the prediction unit 104 to generate coded datato be output.

An integration coding unit 111 performs coding of the outputs from thetile division unit 102 and the filter control information generationunit 103 to generate header code data to be stored in a header. Thegenerated header code data forms a bit stream together with the codeddata output from the coding unit 110 to be output.

A terminal 112 is a terminal from which the bit stream generated by theintegration coding unit 111 is output to the outside.

The coding processing of the image in the image coding apparatusillustrated in FIG. 1 will be described below. According to the presentembodiment, a configuration is adopted in which moving image data isinput in units of frames, but a configuration may of course be adoptedin which still image data for one frame is input.

The image data for one frame which is input from the terminal 101 isinput to the tile division unit 102. The image data input at theterminal 101 is set as image data for one frame which has been combinedby rearranging N pieces (N 3) of images. The arrangement of the N pieces(N 3) of images also includes an arrangement of rotated images.According to the present embodiment, information indicating how manyimages are arranged and combined in what manner may also be obtained.

The tile division unit 102 determines a tile division method for theinput image and divides the input image data into units of tiles inaccordance with the determined division method to be output to theprediction unit 104. That is, in addition, the determined tile divisionmethod is output to the filter control information generation unit 103and the integration coding unit 111 as tile division information. Whilethe determining method for the tile division is not particularlylimited, a feature of the input image may be used, the informationindicating how many images are arranged and combined in what manner asdescribed above may be used, and the determining may be performed by aninput from the user. According to the present embodiment, descriptionswill be provided while it is assumed that the input image data for oneframe is divided along boundaries of N pieces of combined images, andimage data including M pieces (M 2) of boundaries of the tiles isconstituted by N pieces (N 3) of tiles. For example, it is designedthat, in a case where coding of the image as illustrated in FIG. 9B isto be performed, the respective images on front, right, back, bottom,left, and top are associated with individual tiles, that is, coding ofthe image constituted from the six tiles is performed.

Next, the filter control information generation unit 103 determineswhether or not the in-loop filter processing is performed with respectto the respective tile boundaries and outputs the information as thefilter control information to the in-loop filter unit 109 and theintegration coding unit 111. The determining method for the filtercontrol information is not particularly limited, and the feature of theinput image may be used, and the determining may be performed the inputfrom the user. In addition, the filter control information generationunit 103 may perform the determining in accordance with informationrelated to a tile division state which is input from the outside orinternally calculated and information related to a continuity of therespective tile boundaries. For example, in a case where the coding ofthe image as illustrated in FIG. 9B is to be performed, it is determinedthat the coding of the image constituted from the six tiles is performedby the tile division unit 102. In this case, pixel values are continuouson two tile boundaries in the vertical direction (between “front” and“right” and between “right” and “back”) existing in a top part of theimage and one tile boundary in the horizontal direction (between “front”and “bottom”) existing in a left part of the image. On the other hand,the pixels are not continuous on the other four tile boundaries. In theabove-described case, the filter control information generation unit 103outputs information indicating that the in-loop filter processing isperformed with regard to the three tile boundaries where the pixels arecontinuous and information indicating that the in-loop filter processingis not performed with regard to the remaining four discontinuous tileboundary as the filter control information.

The integration coding unit 111 performs coding of the tile divisioninformation and the filter control information to respectively generatethe tile division information code and the filter control informationcode. The coding method is not particularly specified, but Golombcoding, arithmetic coding, Huffman coding, or the like can be used.

The prediction unit 104 divides the image data in units of tiles whichis input from the tile division unit 102 into a plurality of blocks, andthe prediction processing in units of blocks is performed. As a resultof the prediction processing, the prediction error is generated andinput to the transformation and quantization unit 105. In addition, theprediction unit 104 generates the prediction information to be output tothe coding unit 110 and the image reproduction unit 107.

The transformation and quantization unit 105 performs the orthogonaltransformation and the quantization of the input prediction error togenerate a quantization coefficient. The generated quantizationcoefficient is input to the coding unit 110 and the inverse quantizationand inverse transformation unit 106.

The inverse quantization and inverse transformation unit 106 performsthe inverse quantization of the input quantization coefficient toreproduce the transformation coefficient and further performs theinverse orthogonal transformation of the reproduced transformationcoefficient to reproduce the prediction error to be output to the imagereproduction unit 107.

The image reproduction unit 107 refers to the frame memory 108 asappropriate in accordance with the prediction information input from theprediction unit 104 to reproduce the prediction image. Then, the imagedata is reproduced from the reproduced prediction image and thereproduced prediction error input from the inverse quantization andinverse transformation unit 106 to be input to the frame memory 108 andstored.

The in-loop filter unit 109 reads out the reproduction image from theframe memory 108 and performs the in-loop filter processing such as thedeblocking filter in accordance with the block position of the filtertarget and the filter control information input from the filter controlinformation 103. Then, the image on which the filter processing has beenperformed is input to the frame memory 108 again to be stored again.This image on which the filter processing has been performed is used forthe inter prediction in the prediction unit 104 or the like.

The in-loop filter processing according to the present embodiment willbe described by using FIG. 3. Any of the following filter processing isperformed in accordance with the position of the block of the filtertarget and the filter control information. Herein, only the filterprocessing on the tile boundary will be described, but the filterprocessing in the pixel positions other than the tile boundary is set tobe implemented in accordance with the existing image coding method. Inaddition, to facilitate the descriptions herein, the block size is setas a 4×4 block but is not limited to this.

FIG. 3A illustrates an example in a case where deblocking filterprocessing is performed while striding a boundary of two 4×4 blocks. Twoblocks of 4×4 pixels exist inside an outer frame 300, and the filterprocessing is executed with respect to 3 pixels adjacent to tileboundaries of the respective blocks surrounded by a rounded rectangle301, that is, 6 pixels in total. The block on the left side illustratedin FIG. 3A belongs to a tile 0, and the block on the right side belongsto a tile 1. That is, the left and right blocks are blocks belonging todifferent tiles, and a boundary of the left and right blocks is also aboundary of the tiles. This tile boundary is set as a tile boundarywhere the pixel values are continuous as observed in the top part ofFIG. 9B (for example, between “front” and “right”) or the like. In thiscase, since the filter control information indicating that the in-loopprocessing is performed on the tile boundary where the pixels arecontinuous is input from the filter control information generation unit103, all of the 6 pixels indicated by a symbol of ◯ become targets ofthe filter processing, the filter processing is performed with respectto all of the 6 pixels.

On the other hand, FIG. 3B is different from FIG. 3A, and the boundaryof the left and right tiles is set as the tile boundary where the pixelvalues are discontinuous as observed in a bottom part of FIG. 9B (forexample, between “left” and “top”) or the like. In this case, since thefilter control information indicating that the in-loop processing is notperformed on the tile boundary where the pixels are discontinuous isinput from the filter control information generation unit 103, all ofthe 6 pixels indicated by a symbol of x do not become the targets of thefilter processing. Thus, the filter processing is not performed withrespect to the 6 pixels.

The deblocking filter has been mentioned according to the presentembodiment, but a similar control may of course be performed on theother in-loop filter processing such as an adaptive loop filter or thesample adaptive offset. In addition, according to the presentembodiment, the filter processing between the left and right blocks hasbeen described, but the similar processing is also performed between thetop and bottom blocks. Furthermore, the example in which the filteringprocessing is performed on the 3 pixels in the respective blocks hasbeen illustrated, but the number of pixels to be processed is notlimited to this. For example, it is also possible to performasymmetrical filtering processing in which the number of pixelscorresponding to the processing targets in the respective blocks differssuch as 3 pixels in the block on the left side and 2 pixels in the blockon the right side.

While returning to FIG. 1, the coding unit 110 performs the entropycoding of the quantization coefficient generated by the transformationand quantization unit 105 and the prediction information input from theprediction unit 104 in units of blocks to generate the coded data. Theentropy coding method is not particularly specified, but Golomb coding,arithmetic coding, Huffman coding, or the like can be used. Thegenerated coded data is output to the integration coding unit 111.

The integration coding unit 111 multiplies the tile division informationcode and the filter control information code generated prior to thecoding processing, the coded data generated by the coding unit, and thelike, and the bit stream is formed. Finally, it is output from theterminal 112 to the outside.

FIG. 6A illustrates an example of the bit stream including the tiledivision information and filter control information after the coding.The tile division information as the tile division information code andthe filter control information as the filter control information codeare included in any one of headers such as a sequence and a picture.According to the present embodiment, as illustrated in FIG. 6A, thecodes are included in the header part of the picture. FIG. 10Arepresents an example a syntax structure of the header part of thepicture including these codes. Since tiles_enabled_flag is a codeindicating the presence or absence of the tile division, and the casehas been described in which the image illustrated in FIG. 9B is input tobe divided into the six tiles according to the present embodiment, thisvalue becomes 1. Descriptions of entoropy_coding_sync_enabled_flag willbe omitted since it is not related to the present embodiment.num_tile_columns_minus1 indicates a value obtained by subtracting 1 fromthe number of tiles in the horizontal direction. According to thepresent embodiment, since the three tiles exist in the horizontaldirection, this value becomes 2. On the other hand, num_tile_rows_minus1indicates a value obtained by subtracting 1 from the number of tiles inthe vertical direction. According to the present embodiment, since thetwo tiles exist in the vertical direction, this value becomes 1.uniform_spacing_flag is a code indicating whether or not the sizes ofthe respective tiles (the numbers of pixels in the vertical andhorizontal directions) are matched with each other, and since the sizesof all the tiles are matched with each other according to the presentembodiment, this value becomes 1. It should be noted that, when thisvalue is 0, that is, in a case where all the tile sizes are not matchedwith each other, coding of column_width_minus1 and row_height_minus1indicating horizontal and vertical sizes of the respective tiles isperformed, but the descriptions thereof will be omitted according to thepresent embodiment. loop_filter_across_tiles_enabled_flag is a codeindicating whether or not the in-loop filter processing on the tileboundary is enabled, but since the in-loop filter processing is enabledon part of the boundaries according to the present embodiment, thisvalue is set as 1.

Coding of the following syntax is only performed when theabove-described loop_filter_across_tiles_enabled_flag is 1.loop_filter_across_tiles_control_flag is a code indicating whether ornot coding of the information is individually performed which indicateswhether or not the in-loop filter processing is performed with respectto each of the tile boundaries. According to the present embodiment,since the acceptance or rejection of the in-loop filter processingindividually needs to be set, this value is set as 1.

Furthermore, coding of the following syntax is only performed when theabove-described loop_filter_across_tiles_control_flag is 1.loop_filter_across_bottom_tile_boundary_enabled_flag is a codeindicating whether or not the in-loop filter processing is performedwith respect to the tile boundary on the bottom side of the tiles.According to the present embodiment, since three tiles (front, right,and back) in a row on the top side including the tile boundary on thebottom side exist, the coding of this code is performed three times.That is, the coding ofloop_filter_across_bottom_tile_boundary_enabled_flag is performed foreach boundary constituted between two tiles on the top and the bottom.Since the tile boundary on the bottom side of the “front” tile is acontinuous boundary, this first value becomes 1 indicating that thein-loop filter processing is performed. On the other hand, since thetile boundary on the bottom side of the “right” and “back” tiles is adiscontinuous boundary, these second and third values become 0indicating that the in-loop filter processing is not performed. Itshould be noted that with regard to the three tiles (bottom, left, andtop) in a row on the bottom side, the coding ofloop_filter_across_bottom_tile_boundary_enabled_flag is not performed.Next, loop_filter_across_right_tile_boundary_enabled_flag is a codeindicating whether or not the in-loop filter processing is performedwith respect to the tile boundary on the right side of the tiles. Thatis, the coding of loop_filter_across_right_tile_boundary_enabled_flag isperformed for each boundary constituted between two tiles on the leftand the right. According to the present embodiment, since four tiles(front, right, bottom, and left) including the tile boundary on theright side exist, the coding of this code is performed four times. Sincethe tile boundary on the right side of the “front” and “right” tiles isa continuous boundary, these first and second values become 1 indicatingthat the in-loop filter processing is performed. On the other hand,since the tile boundary on the right side the “bottom” and “left” tilesis a discontinuous boundary, these third and fourth values become 0indicating that the in-loop filter processing is not performed. Itshould be noted that, with regard to the tiles (back and top) existingin a column on the rightmost side, the coding ofloop_filter_across_right_tile_boundary_enabled_flag is not performed.That is, the coding ofloop_filter_across_bottom_tile_boundary_enabled_flag is performed withregard to the tile boundary inside the image. Similarly, the coding ofloop_filter_across_right_tile_boundary_enabled_flag is performed withregard to the tile boundary inside the image. Then, the coding is notperformed with regard to the boundary of the tiles constituting an outercircumference.

FIG. 4 is a flow chart illustrating the coding processing in the imagecoding apparatus according to the present embodiment.

First, in step S401, the tile division unit 102 determines a tiledivision method for the input image and divides the input image data inunits of tiles in accordance with the determined division method. Inaddition, the determined tile division method is set as the tiledivision information, and the coding of the tile division information isperformed by the integration coding unit 111.

In step S402, the filter control information generation unit 103determines whether or not the in-loop filter processing is performedwith respect to the pixels on the respective tile boundaries and setsthe information as the filter control information, and the coding of thefilter control information is also performed by the integration codingunit 111.

In step S403, the prediction unit 104 cuts the input image data in unitsof tiles into a plurality of blocks and performs the intra prediction orthe inter prediction in units of blocks to generate the predictioninformation and the prediction image data. Furthermore, the predictionerror is calculated from the input image data and the prediction imagedata.

In step S404, the transformation and quantization unit 105 performs theorthogonal transformation of the prediction error calculated in stepS403 to generate the transformation coefficient and further performs thequantization to generate the quantization coefficient.

In step S405, the inverse quantization and inverse transformation unit106 performs the inverse quantization and the inverse orthogonaltransformation of the quantization coefficient generated in step S404 toreproduce the prediction error.

In step S406, the image reproduction unit 107 reproduces the predictionimage in accordance with the prediction information generated in stepS403. Furthermore, the image data is reproduced from the reproducedprediction image and the prediction error generated in step S405.

In step S407, the coding unit 110 performs coding of the predictioninformation generated in step S403 and the quantization coefficientgenerated in step S404 to generate the coded data. In addition, the bitstream is generated while the other coded data is also included.

In step S408, the image coding apparatus performs a determination onwhether or not the coding of all the blocks in the tile is ended andproceeds to step S409 when the coding is ended and returns to step S403by setting the next block as the target when the coding is not ended.

In step S409, the image coding apparatus performs a determination onwhether or not the coding of all the tiles in the frame is ended andproceeds to step S410 when the coding is ended and returns to step S403by setting the next tile as the target when the coding is not ended.

In step S410, the in-loop filter unit 109 performs the in-loop filterprocessing with respect to the image data reproduced in step S406 togenerate the image on which the filter processing has been performed andends the processing.

FIG. 5 is a flow chart illustrating a detail of the in-loop filterprocessing in step S410.

First, in step S501, the in-loop filter unit 109 determines whether ornot the target pixels exist on the tile boundary from the positions ofthe pixels of the in-loop filter target. When it is determined that thetarget pixels exist on the tile boundary, the flow proceeds to S502, andwhen the target pixels do not exist, the flow proceeds to S503. In S501,a meaning of the existence on the tile boundary refers to existence onthe tile boundary inside the image.

In step S502, the in-loop filter unit 109 determines whether or not thetarget pixels existing on the tile boundary become targets of filteringin accordance with the filter control information generated in stepS402. The in-loop filter unit proceeds to S503 when it is determinedthat the filtering is performed and proceeds to S504 when it isdetermined that the filtering is not performed.

In step S503, the in-loop filter unit 109 performs the in-loop filterprocessing with respect to the target pixels.

In step S504, the in-loop filter unit 109 performs a determination onwhether or not the in-loop filter processing on all the pixels is endedand proceeds to step S505 when the processing is ended and returns tostep S501 by setting the next pixel as the target when the processing isnot ended.

In step S505, the in-loop filter unit 109 performs a determination onwhether or not the in-loop filter processing on all the types is ended.The in-loop filter unit ends the in-loop filter processing when theprocessing is ended and returns to step S501 by setting the next type ofthe in-loop filter processing as the target when the processing is notended.

For example, the two of in-loop filter processing such as the deblockingfilter and sample adaptive offset are defined in the HEVC, switching ofthose is executed in this step. Specifically, first, the deblockingfilter processing is implemented while all the pixels are set as thetargets and thereafter switched to the sample adaptive offsetprocessing, and the flow returns to step S501. In addition, the in-loopfilter processing is ended in a case where the sample adaptive offsetprocessing is also ended. According to the present embodiment, the twotypes of in-loop filter processing such as the deblocking filter andsample adaptive offset similar to the HEVC are executed, but the otherin-loop filter processing (such as, for example, the adaptive loopfilter) may of course be executed. In addition, the in-loop filterprocessing order is not limited to these.

With the above-described configuration and operation, particularly instep S410, while it becomes possible to perform the control on theacceptance or rejection of the application of in-loop filter processingfor each tile boundary, whether or not the in-loop filter processing isapplied to the tile boundary can be chosen in accordance with thecontinuity of the tile boundary. As a result, since it becomes possibleto set in a manner that the in-loop filter processing is applied on thetile boundary where the pixels are continuous and the in-loop filterprocessing is not applied on the tile boundary where the pixels arediscontinuous, the image quality can be improved.

It should be noted that, according to the present embodiment, asillustrated in FIG. 6A, the coding of the tile division information andthe filter control information is performed in the picture header part,but the position where the coding is performed is not limited to this.The coding may be performed in the sequence header part of the image asillustrated in FIG. 6B, or the coding may of course be performed in theother position.

It should be noted that, according to the present embodiment, the codingof the bit stream including the structure of the syntax as illustratedin FIG. 10A is performed, but the structure of the bit stream is notlimited to this. For example, the structure of the syntax as illustratedin FIG. 10B may also be included. In FIG. 10A, the coding of the filtercontrol information is performed with respect to the respective tileboundaries, but FIG. 10B illustrates an example of a case where thecoding of the respectively common filter control information isperformed with respect to all the tile boundaries in the horizontaldirection and then all the tile boundaries in the vertical direction.loop_filter_across_horizontal_tile_boundary_enabled_flag is a flagindicating whether or not the in-loop filter processing is performedwith respect to all the tile boundaries in the vertical direction in theimage. That is, loop_filter_across_horizontal_tile_boundary_enabled_flagis horizontal direction control information common to the plurality oftile boundaries in the horizontal direction.

In addition, loop_filter_across_vertical_tile_boundary_enabled_flag is aflag indicating whether or not the in-loop filter is performed withrespect to all the tile boundaries in the horizontal direction in theimage. That is, loop_filter_across_vertical_tile_boundary_enabled_flagis vertical direction control information common to the plurality oftile boundaries in the vertical direction. For example, in a case wherethe image illustrated in FIG. 9D is input, when the former (⋅ ⋅ ⋅horizontal ⋅ ⋅ ⋅ flag) is set as 1 and the latter (⋅ ⋅ ⋅ vertical ⋅ ⋅ ⋅flag) is set as 0, it is possible to realize filtering control withrespect to the tile boundary with the still smaller code amount. Thatis, in the case of the image in which the plurality of tiles arearranged with respect to a certain direction (the horizontal directionor the vertical direction), the flag including the syntax structure ofFIG. 10B is effective in a case where the acceptance or rejection of theapplication of the in-loop filter processing in each of the tileboundary is the same. In addition, as a modified example of FIGS. 10Aand 10B, the acceptance or rejection of the application of the in-loopfilter processing in one of the horizontal and vertical directions ofthe tile boundaries may be set by using the flag of FIG. 10B. Then, theacceptance or rejection of the application of the in-loop filterprocessing may be set by allocating the flag for the respective tileboundaries in the other direction as illustrated in FIG. 10A.

In addition, the syntax structure like FIG. 10C can also be included.FIG. 10C is characterized in that coding ofloop_filter_across_tiles_control_idc is performed as the filter controlinformation. loop_filter_across_tiles_control_idc indicates an indexcorresponding to a particular tile arrangement method. For example,coding of indices in which “1” is associated with FIG. 9B, “2” isassociated with FIG. 9C, “3” is associated with FIG. 9D, and the like isperformed as the filter control information. The syntax structure ofFIG. 10C is effective in a case where arrangements of the six images tobe combined to constitute the image for one frame are defined as severalformats as illustrated in FIGS. 9B to 9D, for example. In the case ofthe syntax structure of FIG. 10C, the image decoding apparatus isrequired to grasp the acceptance or rejection of the application of thein-loop filter processing with respect to the respective tile boundariesin the arrangements in FIGS. 9B to 9D corresponding to the respectiveindices. According to this, while the plurality of tile arrangementmethods are coped with, it is possible to realize the filtering controlwith respect to the tile boundary with the still smaller code amount.

In addition, the filtering with respect to the rectangular tile boundaryis controlled according to the present embodiment, but the controltarget is not limited to this. Filtering with respect to a boundary ofslices that can also adopt a shape other than the rectangle can besimilarly controlled, and the application can also be performed withrespect to new processing units other than the tiles and the slices.

In addition, it is supposed that the tile is rectangular in accordancewith the existing HEVC according to the present embodiment, but theapplication can also be performed when the tiles and other processingunits adopt other shapes such as a triangle.

FIG. 2 is a block diagram illustrating a configuration of the imagedecoding apparatus according to an embodiment of the present invention.According to the present embodiment, the decoding processing of thecoding data generated in the image coding apparatus will be described.All of the respective blocks in FIG. 2 may be constituted as hardware,or part or all of the blocks may also be constituted as software.

A terminal 201 is a terminal at which the bit stream after the coding isinput. A separation decoding unit 202 separates information related tothe decoding processing and the coded data related to the coefficientfrom the bit stream and further separates the coded data existing in theheader part of the bit stream to be decoded. According to the presentembodiment, the tile division information and the filter controlinformation are reproduced by the decoding processing to be output to asubsequent stage. That is, the separation decoding unit 202 performs anoperation opposite to the integration coding unit 111 of FIG. 1.

A decoding unit 203 decodes the coded data output from the separationdecoding unit 202 to reproduce the quantization coefficient and theprediction information.

An inverse quantization and inverse transformation unit 204 inputs thequantization coefficient in units of blocks similarly as in the inversequantization and inverse transformation unit 106 of FIG. 1, performs theinverse quantization to obtain the transformation coefficient, andfurther performs the inverse orthogonal transformation to reproduce theprediction error.

A frame memory 207 is a memory that stores at least the image for oneframe and stores image data of a reproduced picture.

Similarly as in the image reproduction unit 107 of FIG. 1, an imagereproduction unit 205 refers to the frame memory 207 as appropriate inaccordance with the input prediction information to generate theprediction image data. Then, the reproduction image data is generatedfrom this prediction image data and the prediction error reproduced bythe inverse quantization and inverse transformation unit 204 to beoutput.

Similarly as in the in-loop filter unit 109 of FIG. 1, an in-loop filterunit 206 performs the in-loop filter processing such as the deblockingfilter with respect to the reproduction image and outputs the image onwhich the filter processing has been performed.

A terminal 208 outputs the image data on which the filter processing hasbeen performed to the outside.

The decoding processing on the image in the image decoding apparatusillustrated in FIG. 2 will be described below. According to the presentembodiment, the bit stream generated by the image coding apparatus ofFIG. 1 is decoded.

In FIG. 2, the bit stream input from the terminal 201 is input to theseparation decoding unit 202. The separation decoding unit 202 separatesthe information related to the decoding processing and the coded datarelated to the coefficient from the bit stream and further decodes thecoded data existing in the header part of the bit stream. Specifically,the tile division information and the filter control information arereproduced by the decoding processing. According to the presentembodiment, first, the tile division information code and the filtercontrol information code are extracted from the picture header of thebit stream illustrated in FIG. 6A to be decoded. It should be noted thatthe picture header part uses the syntax structure illustrated in FIG.10A in the following explanation. First, the tiles_enabled_flag code isdecoded, and a value 1 indicating that the tile division is used isobtained. Descriptions of entoropy_coding_sync_enabled_flag will beomitted since it is not related to the present embodiment. Next, thenum_tile_columns_minus1 code is decoded, and a value 2 indicating thatthe three tiles exist in the horizontal direction is obtained.Furthermore, the num_tile_rows_minus1 code is decoded, and a value 1indicating that the two tiles exist in the vertical direction isobtained. Subsequently, the uniform_spacing_flag code is decoded, and avalue 1 indicating that the sizes of the respective tiles (the numbersof pixels in the vertical and horizontal directions) are matched witheach other is obtained. Next, the loop_filter_across_tiles_enabled_flagcode is decoded, and a value 1 indicating that the in-loop filterprocessing on the tile boundary is enabled is obtained.

Since loop_filter_across_tiles_enabled_flag is 1, the syntax decoding isfurther continued. The loop_filter_across_tiles_control_flag code isdecoded, and a value 1 indicating that the in-loop filter processing isperformed with respect to the respective tile boundaries is obtained.

Since loop_filter_across_tiles_control_flag is 1, the syntax decoding isfurther continued. Theloop_filter_across_bottom_tile_boundary_enabled_flag code is decoded,and information indicating whether or not the in-loop filter processingis performed with respect to the tile boundary on the bottom side ofeach of the tiles is obtained. According to the present embodiment,respective values 1, 0, and 0 are obtained as the information withrespect to the three tiles (front, right, and back) including the tileboundary on the bottom side. Next, theloop_filter_across_right_tile_boundary_enabled_flag code is decoded, andinformation indicating whether or not the in-loop filter processing isperformed with respect to the tile boundary on the right side of each ofthe tiles is obtained. According to the present embodiment, respectivevalues 1, 1, 0, and 0 are obtained as the information with respect tothe four tiles (front, right, bottom, and left) including the tileboundary on the right side. The thus obtained filter control informationis output to the in-loop filter unit 206. Subsequently, the coded datain units of blocks of the picture data is reproduced to be output to thedecoding unit 203.

The decoding unit 203 decodes the coded data to reproduce thequantization coefficient and the prediction information. The reproducedquantization coefficient is output to the inverse quantization andinverse transformation unit 204, and the reproduced predictioninformation is output to the image reproduction unit 205.

The inverse quantization and inverse transformation unit 204 performsthe inverse quantization with respect to the input quantizationcoefficient to generate the orthogonal transformation coefficient andfurther performs the application of the inverse orthogonaltransformation to reproduce the prediction error. The reproducedprediction information is output to the image reproduction unit 205.

The image reproduction unit 205 refers to the frame memory 207 asappropriate in accordance with the prediction information input from thedecoding unit 203 to reproduce a prediction image. The image data isreproduced from this prediction image and the prediction error inputfrom the inverse quantization and inverse transformation unit 204 to beinput to the frame memory 207 and stored. The stored image data is usedfor the reference at the time of the prediction.

The in-loop filter unit 206 reads out the reproduction image from theframe memory 207 similarly to 109 in FIG. 1 and performs the in-loopfilter processing such as the deblocking filter in accordance with thefilter control information input from the separation decoding unit 202.Then, the image on which the filter processing has been performed isinput to the frame memory 207 again. Since the in-loop filter processingaccording to the present embodiment is the same as the in-loop filterprocessing in the image coding apparatus of FIG. 1, detaileddescriptions will be omitted.

The image on which the filter processing has been performed in thein-loop filter unit 206 is stored in the frame memory 207 again to beeventually output from the terminal 208 to the outside.

FIG. 7 is a flow chart illustrating the decoding processing on the imagein the image decoding apparatus of FIG. 2.

First, in step S701, the separation decoding unit 202 separates theinformation related to the decoding processing and the coded datarelated to the coefficient from the bit stream and decodes the codeddata in the header part to reproduce the tile division information andthe filter control information.

In step S702, the decoding unit 203 decodes the coded data separated instep S701 and reproduces the quantization coefficient and the predictioninformation.

In step S703, the inverse quantization and inverse transformation unit204 performs the inverse quantization in units of blocks with respect tothe quantization coefficient to obtain the transformation coefficientand further performs the inverse orthogonal transformation to reproducethe prediction error.

In step S704, the image reproduction unit 205 reproduces the predictionimage in accordance with the prediction information generated in stepS702. Furthermore, the image data is reproduced from the reproducedprediction image and the prediction error generated in step S703.

In step S705, the image reproduction unit 205 or a control unit that isnot illustrated in the drawing in the image decoding apparatus performsa determination on whether or not the decoding of all the blocks in thetile is ended. When the decoding is ended, the flow proceeds to stepS706, and when the decoding is not ended, the flow returns to step S702while the next block is set as the target.

In step S706, the image reproduction unit 205 or the control unit thatis not illustrated in the drawing in the image decoding apparatusperforms a determination on whether or not the decoding of all the tilesin the frame is ended. When the decoding is ended, the flow proceeds tostep S707, and when the decoding is not ended, the flow returns to stepS702 while the next tile is set as the target.

In step S707, the in-loop filter unit 206 performs the in-loop filterprocessing with respect to the image data reproduced in step S704 togenerate the image on which the filter processing has been performed andends the processing.

Since a flow chart illustrating the detail of the in-loop filterprocessing is the same as FIG. 5 illustrating the processing in theimage coding apparatus, descriptions thereof will be omitted.

With the above-described configuration and operation, it is possible todecode the bit stream, which has been generated in the image codingapparatus of FIG. 1, with which whether or not the in-loop filterprocessing is applied to the tile boundary can be chosen in accordancewith the continuity of the tile boundary.

It should be noted that the bit stream including the tile divisioninformation and the filter control information in the picture headerpart is decoded as illustrated in FIG. 6A according to the presentembodiment, but the coding position of the information is not limited tothis. The coding may be performed in the sequence header part of theimage as illustrated in FIG. 6B, or the coding may of course beperformed in the other position.

It should be noted that the decoding of the bit stream including thestructure of the syntax as illustrated in FIG. 10A has been illustratedaccording to the present embodiment, but the structure of the bit streamto be decoded is not limited to this. For example, the bit streamincluding the structure of the syntax as illustrated in FIG. 10B mayalso be decoded. FIG. 10B illustrates an example of a case where thecoding of the filter control information common to all the tileboundaries in the horizontal direction and all the tile boundaries inthe vertical direction is performed. For example, in a case where thebit stream generated by the coding of the image illustrated in FIG. 9Dby the coding apparatus according to a first embodiment is decoded, avalue 1 is obtained by decoding the former (⋅ ⋅ ⋅ horizontal ⋅ ⋅ ⋅flag), and a value 0 is obtained by decoding the latter (⋅ ⋅ ⋅ vertical⋅ ⋅ ⋅ flag). According to this, it is possible to decode the bit streamin which the coding of the filter control information indicating thatthe in-loop filter processing is performed with respect to all the tileboundaries in the horizontal direction and the in-loop filter processingis not performed with respect to all the tile boundaries in the verticaldirection has been performed with the still smaller code amount.

Furthermore, it is also possible to decode the bit stream including thesyntax structure like FIG. 10C. In FIG. 10C, it is possible to associatethe index indicated by loop_filter_across_tiles_control_idc with aparticular tile arrangement method. For example, it is designed that 1is associated with FIG. 9B, 2 is associated with FIG. 9C, 3 isassociated with FIG. 9D, and the like. According to this, while thegreater variety of tile arrangement methods are coped with, it ispossible to decode the bit stream that realizes the filtering controlwith respect to the tile boundary with the still smaller code amount.

It has been described according to the above-described embodiment thatthe respective processing units illustrated in FIG. 1 and FIG. 2 areconstituted by the hardware. However, the processes performed by therespective processing units illustrated in these drawings may also beconfigured by a computer program.

FIG. 8 is a block diagram illustrating a configuration example ofhardware of a computer that can be applied to the image coding apparatusand the image decoding apparatus according to the above-describedrespective embodiments.

A CPU 801 performs a control on an entire computer by using a computerprogram or data stored in a RAM 802 or a ROM 803 and also executes theabove-described respective processes as being performed by an imageprocessing apparatus according to the above-described respectiveembodiments. That is, the CPU 801 functions as the respective processingunits illustrated in FIG. 1 and FIG. 2.

The RAM 802 includes an area for temporarily storing computer program ordata loaded from an external storage device 806, data obtained from theoutside via an I/F (interface) 807, and the like. Furthermore, the RAM802 includes a work area used by the CPU 801 when various processes areexecuted. That is, the RAM 802 can be allocated as, for example, a framememory or appropriately provide other various areas.

The ROM 803 stores setting data of this computer, a boot program, andthe like. An operation unit 804 is constituted by a key board, a mouse,and the like and operated by a user of this computer so that variousinstructions can be input to the CPU 801. A display unit 805 displaysprocessing results by the CPU 801. In addition, the display unit 805 isconstituted by a liquid crystal display, for example.

The external storage device 806 is a large-capacity storage devicerepresented by a hard disc drive device. An OS (operating system) andthe computer program for the CPU 801 to realize the functions of therespective units illustrated in FIG. 1 and FIG. 2 are saved in theexternal storage device 806. Furthermore, various image data asprocessing targets may be saved in the external storage device 806.

The computer program and the data saved in the external storage device806 are loaded as appropriate to the RAM 802 in accordance with thecontrol by the CPU 801 to become the processing targets by the CPU 801.A network such as a LAN or the Internet and other apparatuses such as aprojection apparatus and a display apparatus can be connected to the I/F807, and this computer can obtain and transmit various information viathis I/F 807. 808 denotes a bus that connects the above-describedrespective units to one another.

The processes of the flow charts illustrated in FIG. 4, FIG. 5, and FIG.7 described above are executed by the CPU in accordance with the programby reading out the program stored in the ROM 803 to the RAM 802.

As described above, according to the present invention, it is possibleto choose whether or not the in-loop filter processing is applied foreach tile boundary. For this reason, the image quality on the continuoustile boundaries in particular can be improved, and it is possible tofurther improve the coding efficiency.

OTHER EMBODIMENTS

The present invention can also be realized by processing in which aprogram that realizes one or more functions of the above-describedembodiments is supplied to a system or an apparatus via a network or astorage medium. Then, one or more processors in a computer of the systemor the apparatus read out the program to be executed. In addition, thepresent invention can also be realized by a circuit (for example, anASIC) that realizes one or more functions.

The present invention is used for the coding apparatus and the decodingapparatus that perform the coding and the decoding of the still imageand the moving image. In particular, the present invention can beapplied to the coding method and the decoding method in which the tiledivision and the in-loop filter processing are used.

The present invention is not limited to the above-described embodiments,and various alterations and modifications can be made without departingfrom the spirit and the scope of the present invention. Therefore, thefollowing claims are accompanied to make the scope of the presentinvention public.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

1. A coding apparatus for performing coding of an image including aplurality of tiles, the coding apparatus comprising: determining meansfor determining, with respect to a plurality of boundaries constitutedby the plurality of tiles, whether or not filter processing is performedon pixels adjacent to the boundaries; and coding means for performingcoding of control information indicating whether or not the filterprocessing is performed on the pixels adjacent to the boundaries inaccordance with a determining by the determining means with respect toat least two of the plurality of boundaries.
 2. The coding apparatusaccording to claim 1, wherein the plurality of tiles are rectangular andconstitute at least two boundaries.
 3. The coding apparatus according toclaim 1, wherein the coding means performs the coding of the controlinformation for each boundary constituted between two tiles.
 4. Thecoding apparatus according to claim 1, wherein the coding means performsthe coding of control information in a horizontal direction indicatingwhether or not the filter processing is performed on pixels adjacent toa boundary in the horizontal direction and control information in avertical direction indicating whether or not the filter processing isperformed on pixels adjacent to a boundary in the vertical direction asthe control information.
 5. The coding apparatus according to claim 4,wherein the control information in the horizontal direction is controlinformation common to a plurality of tile boundaries in the horizontaldirection.
 6. The coding apparatus according to claim 4, wherein thecontrol information in the vertical direction is control informationcommon to a plurality of tile boundaries in the vertical direction. 7.The coding apparatus according to claim 1, wherein the controlinformation is an index corresponding to an arrangement of a pluralityof images combined to constitute an image for one frame.
 8. A decodingapparatus for decoding an image including a plurality of tiles from abit stream, the decoding apparatus comprising: decoding means fordecoding the image; generation means for generating, with respect to aplurality of boundaries constituted by the plurality of tiles, controlinformation indicating whether or not filter processing is performed onpixels adjacent to the boundaries from the bit stream; determining meansfor determining whether or not the filter processing is performed on thepixels adjacent to at least a plurality of boundaries in accordance withthe control information generated by the generation means; andprocessing means for performing the filter processing with respect tothe boundaries where the determining means determines that the filterprocessing is performed.
 9. The decoding apparatus according to claim 8,wherein the plurality of tiles are rectangular and constitute at leasttwo boundaries.
 10. The decoding apparatus according to claim 8, whereinthe generation means generates the control information for each boundaryconstituted between two tiles.
 11. The decoding apparatus according toclaim 8, wherein the generation means generates control information in ahorizontal direction indicating whether or not the filter processing isperformed on pixels adjacent to a boundary in the horizontal directionand control information in a vertical direction indicating whether ornot the filter processing is performed on pixels adjacent to a boundaryin the vertical direction as the control information.
 12. The decodingapparatus according to claim 11, wherein the control information in thehorizontal direction is control information common to a plurality oftile boundaries in the horizontal direction.
 13. The decoding apparatusaccording to claim 11, wherein the control information in the verticaldirection is control information common to a plurality of tileboundaries in the vertical direction.
 14. The decoding apparatusaccording to claim 8, wherein the control information is an indexcorresponding to an arrangement of a plurality of images combined toconstitute an image for one frame.
 15. A coding method for performingcoding of an image including a plurality of tiles, the coding methodcomprising: a determining step of determining, with respect to aplurality of boundaries constituted by the plurality of tiles, whetheror not filter processing is performed on pixels adjacent to theboundaries; and a coding step of performing coding of controlinformation indicating whether or not the filter processing is performedon the pixels adjacent to the boundaries in accordance with adetermining in the determining step with respect to at least two of theplurality of boundaries.
 16. A decoding method for decoding an imageincluding a plurality of tiles from a bit stream, the decoding methodcomprising: a decoding step of decoding the image; a generation step ofgenerating, with respect to a plurality of boundaries constituted by theplurality of tiles, control information indicating whether or not filterprocessing is performed on pixels adjacent to the boundaries from thebit stream; a determining step of determining whether or not the filterprocessing is performed on the pixels adjacent to at least a pluralityof boundaries in accordance with the control information generated inthe generation step; and a processing step of performing the filterprocessing with respect to the boundaries where it is determined in thedetermining step that the filter processing is performed.